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An imaging instrument (40) includes a compact hand-held housing (44) having an electronic imaging element (72) supported within 
a housing, and a plurality of interchangeable instrument heads (52. 56, 60, 64) separately attachable to the housing. Each of the yistrament 
heads Includes an optical system disposed in alignment with the electronic imaging element along an instrument viewing axis. Preferably, 
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then transferred using a receiving cradle (186) to a computer (210) which contains software which organizes the stored data for further 
processing. In a preferred example, the audio files can be transcribed through a network utilizing voice recognition software. 
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COMPACT IMAGING INSTRUMENT SYSTEM 
FIELD OF THE INVENTION 

This invention relates to the field of medical instrumentation, and in particular to a 
data management system utilizing at least one examination instrument capable of storing 
different forms of data. The system allows upload and download of captured data between 
5 the examination instrument(s) and a computer network to create and maintain records and 
reports. 

BACKGROUND OF THE INVENTION 

A number of hand-held diagnostic or examination instruments are known in the 
medical field for examining a patient, such as those commonly used during routine physician 

10 office visits. These instruments include, among others, skin surface microscopes which are 
used for diagnosing skin disorders, otoscopes permitting examination of the ear canal and 
tympanic membrane, and ophthalmoscopes for examining the eyes. Each of the above 
instruments have uniquely inherent features to allow an effective examination of the area of 
interest. Skin surface microscopes, for example, include a distal optical el^ent having a 

15 relatively large diameter (e.g. approximately 15 nun) for direct placement onto a wart, lesion, 
or other skin disorder. Otoscopes, on the other hand, include a fiusto-conical insertion 
portion having a relatively small diameter (4 mm) optic along with an overlaying safety 
speculum, which prevents insertion beyond a predetermined distance into the ear canal. 

It has since become desirable for a patient to be able to witness a primary care or other 

2 0 examination along with the physician. Therefore, yideoized versions of the above diagnostic 
instruments have been developed, such as those described in U.S. Patent No. 5,363,839, 
issued to Lankford, U.S. Patent No. 5,239,984, issued to Cane, et al, and U.S. Patent No. 
4,947,245, issued to Ogawa, et al. In each of the referenced instruments a miniature video 
camera, such as a CCD or other electronic sensor, is positioned either within the interior of 

25 the instrument or adjacently coupled thereto. The electronic sensor includes a light receiving 
surface or substrate which receives a focused optical image of a target of interest through a 
specifically designed viewing system, such as a rod lens, objective or other form of lens, 
typically positioned in the distal end of the instrument. 
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A separately disposed light box, or other source of illumination, provides white light 
through a sheathed cable which is tethered to the proximal end of the instrument. The cable 
includes an optical fiber bundle for directing the light specifically to the distal tip of the 
instrument, as well as electrical conductors for powering the electronic sensor. The electronic 
5 sensor, in turn, creates an analog or digital electrical signal which is remotely transmitted to a 
processor containing appropriate circuitry for then converting the transmitted electrical signal 
into a video monitor-ready (PAL, NTSC) format. The processed video signal is then 
separately displayed on a remote monitor. The use of videoized systems has become 
increasingly popular and has since taken on the term "telemedicine". 
10 Telemedicine systems, like those described above, are quite expensive, with each 

system requiring a separate diagnostic instrument, along with dedicated cabling, light box, 
signal processor and video peripheral device(s). In addition, each system also requires a 
significant space allotment, posing a separate problem in that space is at a premium in 
physician's offices and other environments where such systems would be typically be used. 
15 It is therefore desirable to provide a telemedicine system which is capable of performing and 
displaying multiple forms of examinations. 

It is another perceived desire in the field to make such telemedicine systems portable. 
In fact, it is highly desirous to allow examinations to take place outside the "normal doctor's 
office". Along with this need, is a similarly recognized need to allow portions of the system 
2 0 to be compactly arranged without the need for separate peripherals or connecting devices. 

Improved organization of patient records is yet another current need in the medical 
field. To date, creation and maintenance of patient files has been largely a manually managed 
. activity. Data which can form a part of the overall patient record, however, can take on a 
number of different forms. For example, it has been known that data can be accumulated in a 

2 5 nimiber of forms, particularly with the advent of telemedicine involving image capture, 

portions of which might form part of a patient chart or record. 

In addition, ph3^icians, such as family practitioners, surgeons, etc., invariably record 
notes during a patient visit and examination. In some instances, of course, the physician may 
write information directly into the patient's file. The usual practice, however, is to record 

3 0 events of an examination using a hand-held recording device. The taped notes are then later 

transcribed and then added to the patient's file. Throughout the course of a single day. 
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however, it is possible that a physician may see as many as 40 patients. This kind of volume 
makes the task of compiling and transcribing notes difficuU, or at a minimum time 
consuming, either for the physician or for the physician's staff. Reference is herein briefly 
made to Fig. 12 and 13, broadly depicting a transcription procedxire in accordance with the 
5 prior art. 

More particularly, and prior to a patient encounter, a patient chart is taken from the 
office files and given to the physician. The physician then examines the patient, adding his or 
her notes to the file and dictating as needed during the course of examination, typically using 
a hand-held tape records:. The physician identifies the patient and adds a time and date 

1 0 stamp at the start of each dictation session. Usually, a single tape will contain dictation 

relating to a plurality of patient encounters over the course of a typical working day or shift. 
Often the dictation will be done at the end of the day, when details of the patient visit are 
sketchy. The doctor must usually rely upon memory, and whatever notes made during the 
course of examination. As noted, however, a doctor will often see many patients during the 

1 5 day, obscuring the details of a specific visit. 

The tape is then sent to a transcriptionist, who listens to the tape, as best understood, 
and manually types the chart notes. The notes are either into typed into a computer record or 
onto paper for each patient record on the tape. In any event, a copy of the chart notes are then 
printed and forwarded to the physician for review. The physician fills in any data which 

2 0 could not be successfiiUy interpreted by the transcriptionist, and otherwise edits the chart 

notes which are then typed in accordance with the corrections. After again reviewing the 
chart notes, as necessary, the physician signs off the transcribed record. The record is then 
added to the patient's file. The creation of patient records or reports incorporating several 
different types of data, including audio and video data files, is even more difficult. 
25 A number of potential problems can occur from the above procedure. First, if the 

tape(s) is faulty or lost, the physician will be required to create the chart records Srom 
memory and/or consultation of any written notes which may have been taken during the 
examination. Asimilarproblempccursifthetapeisprematurely and accidently erased. The 
audio tape is the primary source of information, which both the transcriptionist and the 

3 0 physician must rely upon for both creating the draft chart notes and for reviewing purposes. 

the end result is a heightened probability that the records will be incomplete or inaccurate. 
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In a similar vein, the printed notes could also be lost or misplaced, potentially 
delaying the reviewing process. Delays obviously will increase the probability that incorrect 
or incomplete records will be generated. If known video diagnostic instruments such as those 
shown in Fig. 1 are used, the data obtained from each must also be labeled and separately 
5 attached to the file. Based on the amount of time taken, it could be difficult to correctly 
place this data with the transcribed data, if any. As should be apparent, a myriad of different 
combinations using various types of data are possible. 

To date, though there are known transcripting apparatus available, none conveniently 
combine audio data with other foims of collected data, such as captured images, sketches by 
10 the physician, or other patient or related data obtained from otha: instruments to be retained 
and used in compiling and assembling complete examination records which can then be 
effectively stored and maintained. 
SUMMARY OF THE INVENTION 

Accordingly, it is a primary object of the present invention is to improve the present 
1 5 state of the art of medical examination systems. 

It is yet another primary object of the present invention to provide a data management 
system, for medical or other records, such as inspection, quality control, inventory, etc. using 
an instrument which is capable of capturing and storing multiple forms of data input. This 
data can then be effectively transferred into* a central network capable of linking the data into 

2 0 accumulated data records which can be updated and maintained automatically. 

Yet another primary object of the present invention is to provide an instrument 
capable of storing various forms of data (i.e.: a multimedia instrument), that can be adaptively 
interconnected with a plurality of output devices to allow transfer and subsequent processing 
of a plurality of stored data inputs. 
25 It is yet another primary object of the present invention to provide a medical records 

(data) management system in which a plurality oif captured audio files can be subsequently 
transferred and transcribed remotely. The transcribed results can then be transferred directly 
into a patient record along with other data forms relating to a specific patent encounter. The 
entirety of the data can be stored and/or output into a convenient format which can be printed 

3 0 and added to a patient file. 
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Therefore and according to a preferred aspect of the present invention, there is 
provided a method for creating and maintaining records containing at least image, audio data 
relating to a plurality of subjects using a hand-held multimedia examination instrument 
having means for capturing various forms of data, the method comprising the steps of: 
5 capturing image data relating to at least one subject of interest using said hand- 

held multimedia instrument; 

capturing audio data corresponding to at least one subject of interest using said 
muhimedia instrument; 

capturing control data relating to said at least one subject of interest using said 
10 multimedia instrument; 

transferring the captured audio and image data from said multimedia 
instrument to a computer site; 

storing said data in a database associated with said computer site; 
transferring audio data from said computer site to a transcription site; 
15 transcribing said audio data at said transcription site into text-readable data; 

transferring said text-readable data to said computer database for storage; and 
incorporating said text-readable data with associated image data as part of a 

: data record. 

More preferably, the hand-held multimedia instrument is part of an overall data or 

2 0 records management system. According to a preferred embodiment, the instrument can be 

interconnected to a receiving cradle or docking station having means for allowing data 
transfer between the instrument and an external source to allow transfer of audio, video and 
other data files stored in the instrument or the external source which can, for example, be part 
of a single computer or computer network. In this manner, protocols, operating instructions, 
25 or data from other instruments can be transferred directly to the multimedia instrument 

according to one embodiment and data and the like can be transferred from the instrument. 

In accordance with a preferred embodiment, data is transferred to an intermediate or 
local PC utilizing software which arranges the data mto a scripted template, such as a patient 
chart of convenient architecture. The template includes allocation for voice, video, 

3 0 aimotation and other data and can be stored in a local database. The local database, would for 

example, contain patient files for a specific physician's office. 
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In addition, the transferred voice or WAV files can be further transferred into a 
network including a data center (e.g. a server) utilizing a global database for tying in a 
plurality of similar muhimedia or other suitable devices. According to a preferred 
embodiment, the data center can receive and extract a plurality of raw voice data files Jtom a 
particular instrument, direct the files remotely for transcription and subsequently report all 
data of a patient encounter, including transcription or consultation data taken fix)m voice 
inputs, back to the local physician in a suitable record format. 

According to a preferred feature, the described data management system utilizes 
software capable of discriminating a captured video image from known ID or 2D bar code 
symbology or for pattern recognizable data. This allows the multimedia instrument to tag 
data files automatically without requiring s^arate manual input fi-oin the user. 

Accordmg to yet another preferred embodiment, the multimedia diagnostic instrument 
includes individual instrument heads, each head including separate and imique viewing optics 
which focus an optical image onto a contained electronic imager, the imager being preferably 
situated adjacent the fipont face of the instrument body. 

According to yet another preferred aspect of the present invention^ there is described a 
record management system comprising: 

at least one examination instrument including a plurality of instrument heads, wherein 
at least one of said heads and said instrument body include an optical system for directing an 
image onto an electronic sensor disposed in said instrument and display means for displaying 
at least one directed image and data capture means for capturing audio and video data; 

means for transferring data files Scorn said instrument to a processing means, said 
processing means including means for transcribing notes from audio data contained in said 
files; and for accumulating data into a record format. 

According to a preferred aspect, there is described a method for transcribing a 
plurality of record notes, comprising the stq)s of: 

storing a plurality of data in several forms using a multimedia instrument, said 
instrummt having digital camera means and display means contained therein^ as well as 
means for taking audio data corresponding to a displayed video image; 
transferring data Srom said instrument into a database; 
extracting audio data fi:-om said transferred data; 



wo 99/42029 PCTAJS99/03S59 

7 

moving the audio data files to a central processing station; and 
using voice recognition software to process the transcription notes, the software 
preferably being able to recognize and utilize learn technology based on a given voice being 
recognized for processing; 
5 creating transcriptions which can be associated with a patient file having at least one 

video image attributed thereto. 

An advantage of the present invention is that a telemedicine system is provided which 
allows multiple types of examination to be performed in a simple and efficient manner using 
a single instrument body and interchangeable instrument heads. 
1 0 Another advantage of the present invention is that multiple instrument heads can be 

selectively and simply interchanged with a single instniment body to provide versatility and 
to provide the advantages of multiple videoized systems without a significant impact beyond 
that of a single dedicated system. Moreover, the instrument is portable, meaning that 
examinations are not confined to a dedicated location, such as a doctor's office. 
15 Still another advantage of the present invention is that the described system allows 

multiple examinations to be performed in a space envelope which is smaller than 
conventionally known videoized systems. The instrument also includes an integral display 
and means for compactly storing a series of images, or of displaying real or stored images and 
playback of captured audio-related data. This capability allows the physician to more 
2 0 efficiently improve the c^abilities of the office. In addition, the instrument is preferably 
linkable to a PC, a PC network or other peripherals capable of using data retrieved firom the 
instrument. Yet, the physician or other user of the instrument can use the videoized 
instrument fi-om literally any location without restriction, for example, to an office setting. 

Still another advantage of the present system is that numerous types of data including 

2 5 imaging data, audio data, and annotation data can be easily stored, transferred, and utilized. 

This storage allows the creation of a ^'multimedia" data file and allows efficient creation and 
maintenance of records provided in a useful format which incorporates each data type within 
the confines of a specific record. 

Yet another advantage is that the above described system can be easily adapted into a 

3 0 multimedia data management system. According to one specific example, a transcription 

service can be created allowing audio data captured and stored by the instrument(s) to be 
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added into a computer network having voice processing software using a cradle or dictating 
station which is tied to a local PC which can be linked into the network. As a result, doctors 
can review records more quickly because the files incoiporate image data, allowing the 
physician to recognize patients faster and recall particular conditions and physiology. An 
5 immediate benefit occurs when dictation occurs later after a number of separate patient 
encounters, and when transcribed notes are reviewed days later. 

These and other objects, advantages, and features will be described in the following 
Detailed Description of the Invention which should be read in conjunction with 
accompanying drawings. 
10 BRTEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a fix>nt isometric view of a medical diagnostic instrument system in 
accordance with the prior art; 

Fig. 2 is a side partial view of a multimedia examination instrument having 
interchangeable instrumoit heads made in accordance with a preferred embodiment of the 
15 present invention; 

Fig. 3 is a fi-ont elevational view of the multimedia instrument of Fig. 2; 

Fig. 4 is a side partially sectioned view of the multimedia instrunieht as taken through 
Ime 4-4 of Fig. 3; 

Fig. 5 is a rear elevational view of the multimedia instrument of Figs. 2-4 
2 0 Fig. 6 is a side perspective view of the instrument of Fig. 5, showing the sliding cover 

being moved to an open position to reveal a touch sensitive display; 

Fig. 7 is a partial rear view of the touch sensitive video display of Fig. 6; 
Fig. 8 is a block diagram of a preferred system architecture for a multimedia 
examination instrument in accordance with the present invention; 
2 5 Fig. 9 is a systematic diagram of an examination instrument as used with a receiving 

cradle capable of transferring data between the diagnostic instrument and a central data 
network; 

Fig. 10 is a schematic diagram showing a portion of the data management system and 
in particular data transfer between a network center and local PC stations; 
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Fig. 1 1 is a schematic diagram of the portion of the data management system of Fig. 
10 illustrating multiple alternate forms of data transfer between the local PC stations and the 
network center as contemplated by the present invention; 

Figs. 12 and 13 are flow charts representative of prior art methodology pertaining to 
5 transcription of audio medical records; 

Fig. 14 is a simplified flow chart of a transcription/report aspect of the data 
management system of Figs. 1 0 and 11; 

Fig. 15 is an enhanced diagrammatic view of a transcription/report transcription 
procedure of Fig. 14; 

1 0 Fig. 1 6 is a flow chart of a patient encounter utilizing the data management system of 

Fig. 15; 

Fig. 17 is a schematic chart illustrating multiple altemate forms of data transfer 
between remote transcription sites and the network center for the data management system of 
Figs. 10, 11, and 14-16; and 
1 5 Fig. 1 8 is a sample report created using the data management system of the present 

invention. 

DETAILED DRSCRIPTIQN OF THE INVENTION 

The following discussion describes the present invention according to certain specific 
embodiments and more particularly to a physician's office enviroimient. As will be apparent 
2 0 from the following discussion, however, there are many other modifications and variations 
which can be employed by those of skill in the field embodying the concepts which are 
described herein. For example, the above system could similarly be used in a hospital 
emergency room, HMO or other environment in which a plurality of patient encounters take 
place on a daily basis. In addition, and though the presently described embodiments relate 

2 5 specifically to the medical field, it will be readily apparent to one of sufficient skill in the 

field that other suitable fields utilizing numerous forms of data input and reporting for 
varying numbers of samples including, but not limited to manufacturing, quality control, 
inspection, engineering, and inventory can effectively utilize the inventive concepts presented 
herein. 

3 0 Turning to Fig. 1, there is shown a video medical diagnostic instrument system in 

accordance with the prior art. The instrument system 10 includes a medical diagnostic 
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instrument 14, in this instance an endoscope (ie: a video laparoscope is shown) defined by an 
elongate instrument body 16 having a distal end 18 and an opposite proximal end 17 attached 
to a handle section 20. An electronic sensor or element (not shown), such as a CCD (charge 
coupled device), is disposed within the instrument body 16 and receives an optical image of a 
target of interest through an imaging system, such as a relay, objective or other known lens 
system (not shown) in a conventional manner. The electronic sensor includes support 
electronics which convert the optical signal into an electrical signal which is transmitted 
along a sheathed cable 22 depending from the proximal end 24 of the handle section 20. 

According to the above-described system, a video processing module 28 forms the 
proximal end of the sheathed cable 22, the module containing processing electronics for 
converting the transmitted electrical signal into a video monitor-ready (PAL, NTSC, etc.) 
signal. The video processing module 28 is attached into a receiving cavity 29 of a 
light/power box 32 containing a high output light source, such as an arc lamp (not shown) or 
other source of white light. The light from the high-intensity light source is transmitted from 
the light box 32 through an optical fiber bundle (not shown) contained within the sheathed 
cable 22, and guided into the diagnostic instrument body 16 to the distal end 18 thereof The 
light/power box 32 also serves to furnish power to the diagnostic instrument 14 through 
electrical connectors, also contained within the sheathed cable 22, the power/light box being 
operated by a control panel 30. 

In use, a processed video signal of the target of interest is displayed by an 
interconnected video monitor 34 which is connected to the light/power box 32 to allow 
viewing, by a physician and patient(s). Other peripheral devices, (not shown) such as a video 
printer, a video tape recorder, a PC, etc., can also be substituted into the above described 
instrument system. 

The above diagnostic instrument system 10 introduces a number of discrete 
components and requires a significant spatial footprint typically restricting the use of the 
system to a dedicated area, such as a physician's office, an emergency room, etc. Though the 
above laparoscopic system is dedicated to a particular target of surgical interest, (e.g., the 
abdominal cavity), other types of diagnostic instruments, such as otoscopes, colposcopes, and 
demiatoscopes, among others, are required for performing other types of examinations that 
are typically done during a patient visit. That is, it is not uncommon that a variety of 
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different examinations, (ear, eye, throat, skin) be performed in a single family practitioner 
visit. The ability to electronically capture and archive images for each type of examination 
would be desirable, allowing the patient and the physician to both view a target of interest, 
but as noted above, typically a separate dedicated system is required for each instrument. 
5 MULTIMEDIA DIAGNOSTIC INSTRUMENT 

Referring now to Fig. 2, a diagnostic instrument system 40 according to a preferred 
embodiment of the present invention comprises a compact diagnostic instrument 42 including 
a housing or body 44 having a front interface 48 with means for allowing selective releasable 
attachment thereto of a plurality of instrument heads. According to this embodiment, the 

1 0 instrument heads include a general purpose instrument head 52, a dermatological instrument 
head 56, a high magnification instrument head 60, and an otological instrument head 64. 
Other types of instrument heads, such as for ophthalmoscopic use and employing optical 
systems such as described in commonly assigned U.S. Patent Nos. 4,526,449 and 4,998,818, 
for example, can also be utilized. 

15 In brief and referring to Figs. 2-4, each of the instrument heads 52, 56, 60, and 64 

include a latching member 70 which engages a cavity 74 provided in the front interface 48 to 
allow releasable engagement with the diagnostic instrumCTt 42. Some or all of the latching 
members 70 include electrical contacts 82 which engage corresponding contacts 80 extending 
from the front interface 48 to power an illumination system contained in an instrument head. 

20 In addition, each of the instrument heads include unique optical systems to enable viewing of 
a target of interest when assembled. Specific details relating to the instrument heads and the 
latching mechanism are provided in USSN 09/052,570. 

Referring to Figs. 3 and 4, the diagnostic instrument 42 for purposes of the described 
system 40, Fig. 2, is a compact hand-held digital camera having a defined interior 68, shown 

2 5 partially in Fig. 4. The interior 68 is appropriately sized to retain a plurality of components 

including an electronic imaging element 72, such as a charge coupled device (CCD) having 
related processing circuitry or CMOS having substantially all of the video processing 
disposed directly on the chip, disposed adjacent a window 76 or clear covering at the front 
interface 48 provided in the instrument housing 44. 

3 0 The instrument 42 further includes a controller, such as a digital video processing 

engine 78, Fig. 8, having sufficient memory and programmable logic contained within the 
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interior 68 of the instrument housing 44 and interconnected to the retained components, 
including an integral touch-sensitive TFT liquid crystal display (LED) 84, provided on the 
rear side 86 thereof. Alternately, an eyepiece type of display, shown diagrammaticaUy iii Fig. 
8, could be used. The processed digital video signal is outputted to the display 84 by a 
5 composite video generator 152, Fig. 8, for viewing by the user. A protective cover 85, 
slidingly attached to the rear side 86 of the housing 44 by known means, allows selective 
access to the display 84. Preferably, the rear-side 86 of the instrument housing 44 is angled, 
as shown in Fig. 4 by reference numeral 90, relative to the vertical axis 92 and orthogonal to 
the viewing axis 93 of the instrument 42 to facilitate viewing of the display 84 for the user. 
0 According to the embodiment, an angle, represented in Fig. 4 as -A-, of approximately 15 
degrees is suitable. 

Referring to Figs. 2, and 6, the architecture of the digital video processing engine 78, 
Fig. 8, allows various forms of data to be captured and stored in conjunction with image 
(video) information. An internal condenser microphone 94 disposed on the top exterior of the 
instrument housing 44 allows audio information to be captured and stored selectively into the 
buffer memory of an audio processor 148, while an integral speaker 96 disposed on the rear 
side 86 allows playback of the stored audio information in conjunction with a stored video 
image. Audio data that has been captured in the internal buffer of the audio processor 148 is 
transferred to the digital video processing engine 78, which appends the audio data to an 
electronic file containing the desired image and aimotation data, along with control data. This 
file is saved in a memory card 167, with memory interface 166 managing the flow of data 
between the memory card and the digital video processing engine. 

Referring to Fig. 6, a plurality of control switches located on the exterior of the 
instrument housing 44 includes a POWER ON/OFF switch 98, as well as a 
RECORD/PLAYBACK switch 100 controlling the audio recording and playback features of 
the camera. A series ofindicating lamps are also provided, more specifically a power lamp 
102, a ready lamp 103, and a recording lamp 104. 

Referring briefly to Fig. 7, the TFT display 84 according to the present device 
includes a main window 106 and a plurality of selectable keys disposed about the periphery 
thoeof, including a key for accessing a main menu 108, an ENTER key 1 10, a CANCEL key 
112, and a DELETE key 114. Keys 116 are also provided to allow scroUing in either vertical 
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direction. The main window 106 can be selectively divided into separate image fields for 
allowing multiple stored digital images to be displayed simultaneously, and to allow 
annotation relating to a displayed image (s). Exemplary image fields 118 and 120 and an 
annotation field 122 are shown in Fig. 7, though preferably the programmable architecture of 
5 the digital video processing engine 78, Fig. 8, allows hterally any number of separate fields 
to be made available. For example, a plurality of miniature captured images (not shown) can 
be displayed in a sequential manner as a slide show presentation on the main window 106. A 
stylus pen (not shown) selectively allows notes to be added in the illustrated annotation field 
122. The notes are also stored into the internal memory card 1 67. 

1 0 The described digital camera used as the diagnostic instrument 42 includes other 

salient and specific features relating to image capture, such as programmed auto-exposure 
control, including an electronic single-frame shutter and automatic gain control. The specific 
teachings of these features do not specifically form a part of the present invention. Therefore, 
no further discussion is required. 

15 Preferably, the instrument housing 44 includes a compartment 124, Fig. 5, accessible 

from the rear side 86 thereof for retaining a set of rechargeable batteries (not shown) for 
powering the instrument 42. Alternately, a separate ad^ter cord (not shown) can supply 
power fix>m a suitable AC outlet (not shovm). 

Referring to Figs. 2-7 and in use, one of the instrument heads is attached to the firont 

2 0 interface 48, aligning the optics of the instrument head with the imaging element 72. 

Activation of the diagnostic instrument 42 using POWER ON switch 98 activates the imaging 
element 72 and processing circuitry so as to allow a real time video image to be viewed on 
the TFT display 84. The viewed image can be selectively captured using a shutter release 
button (not shovm) provided on the instrument housing 44 (not shown), causing the image to 
25 be stored into the memory card 167, Fig. 8. Activation of the switch 100 allows the 

microphone 94 to be enabled to allow audio data to be captured corresponding to the video 
image which is being currently displayed. The instrument 42 includes a MENU feature 
controlled by the audio processor 148, Fig. 8, which allows the length of the sound clip to be 
controlled. Alternately, other modes are provided for recording sound without use of the 

3 0 video capture mechanisms, if desired. Audio data is stored in a .WAV format, though other 

formats with varying degrees of compression may also be used. In the present ^bodiment. 
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approximately 17 minutes of sound data memory are provided though this quantity can easily 
be varied. 

Preferably, the programmable architecture of the digital video processing engine 78 of 
the presently described instrument 42 also includes an internal calendar, including a date and 
5 time stamp, which automatically provides an entry which is stored with each corresponding 
video and/or audio image captured. 

Image data in memory card 167 are stored in the presently described instrument 42 
using JPEG compression to reduce the amount of memory they consume. Image quality can 
be enhanced by adjusting a menu setting in order to produce either high quality photographs 
10 or normal (compressed) photographs which increases the compression ratio and reduces the 
amoimt of memory needed to store each photograph. In the described camera, the high 
quality mode allows 66 images to be stored using a compression ratio of 10:1 and 132 
photographs to be stored using a compression of 20:1 in the normal mode. 

As described in greater detail below, the present multimedia instrument 42 includes a 
1 5 serial, SCSI or other form of data transfer port so as to allow selective intercoxmection to a 
computer or more preferably to a docking station or cradle 126, Fig. 9, which is similarly 
linked to a local PC site 190, Fig. 9, or PC network. Altemately, the stored data (video, 
armotative, audio, control, etc) can also be transmitted to a video printer, or other suitable 
peripheral device. Data can also be transferred from the instrument 42 by removing and/or 
2 0 reinserting the memory card 167, as needed. 

As noted. Figs 2-7 relate to a particular multimedia instrument 42 useful for the 
present embodiment. The digital camera depicted in the present embodiment is a "COOLPIX 
300" sold by the Nikon Corporation, though it will be apparent that other known compact 
digital cameras, such as the Kodak DC-260, manufactured and sold by the Eastman Kodak 

2 5 Company, or any digital camera having similar or other features can be similarly configured 

for use in the described diagnostic instrument system. 

Reference is now made to Fig. 8 which dqiicts a more generalized architecture of a 
multimedia instrument in accordance with the present invention. The digital video processing 
engine 78 fomas the central hub of the instrument which is interconnected to a series of 

3 0 modules, including an imaging module 130, an audio module 140, a video/display module 

150, a communications interface module 160 and an illumination control module 170. The 
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imaging module I30 includes at least one lens element, shown diagrammatically as 132, 
which can be located in the instrument and/or instrument head and is aligned with the 
electronic imager 133 used for directing an optical signal to an electrical signal through a 
correlated double sampler (CDS) 134 and an A/D converter 136 for creating a digital signal 
which is stored into the buffer memory of the digital video processing engine 78 using a 
timing generator 138 to control shuttering and signal transfer from the imager 133. 

The audio module 140 includes a microphone 142 and speaker 144, each 
interconnected through appropriate converters 146 to an audio processor 148 which is tied by 
known means to the digital video processing engine 78, which includes logic to correspond 
audio data with video data from the imaging module 130 for storage. 

The video/display module 150 allows display of videoized output provided through a 
composite video generator 152 to either a local TFT or eyepiece display 154 or to an external 
video monitor 1 56. All of the above data can be uploaded or data can be downloaded to the 
instrument through the communications interface module 160, such as through use of RS232, 
USB, or a serial port 163, 161, and 162 respectively. Other data, such as vital statistics 
(patient ID, height, weight, age, etc) can be directly input through a keypad 164, if provided, 
while other data, such as analog or digital data from a separate medical instrument, operating 
instructions, scripts, and ttie like can also be selectively added or extracted using a memory 
interface 166, including memory cards 167. Original instructions or protocols can be tied into 
the digital video processing engine's processor (not shown) through embedded programming 
172. 

The illumination control module 170 can be tied through the latching mechanism as 
described in USSN 09/052,570 to the individual instrument heads to the power source 174 
(batteries, AC, or other) or a separate illumination control can be provided. 
DATA MANAGEMENT SYSTEM 

With the background of the examination instrument portion of the above system 
completed, reference is now made to Fig. 9 which illustrate a schematic diagram of a portion 
of a preferred embodiment of a data/records management system in accordance with the 
present invention. In brief, a diagnostic instrument 1 80, similar to the instrument 42 
described above, and including the multimedia features of Fig. 8, used in conjunction with a 
receiving cradle 186 to facilitate data transfer in order to allow uploading of audio, video and 
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Other stored data. The cradle 186 is interconnected to a local computer station or site 190, 
such as found in a physician's ofBce, which incorporates the uploaded data into a local 
database 192. In the described embodiment, audio (WAV) files are arranged with 
corresponding video and other stored data in a template which forms the basis for a patient 
record data sheet, such as shown in Fig. 1 8, the individual files being separated by identifiers 
(for example, patient and doctor names) as described in greater detail below. 

The instrument 180 includes a housmg 184 having an integral display 183 and an 
interface 181 which uses a plurality of selectively interchangeable instrument heads (not 
shown in this view) like those previously referred to m Fig.2. As in the preceding, each of 
the instrument heads (not shown) includes a specific optical system allowing an optical image 
of interest to be focused onto a contained electronic imaging element. Support electronics 
(not shown) converts the optical signal into a captured video signal which is displayed on 
display 183; A microprocessor (not shown) contains programmable logic which allows a 
real-time image to be continuously displayed and also allows a predetermined number of 
images to be captured and stored into memory, selectively or otherwise, along with 
corresponding audio and/or annotation data added using an integral microphone. The display 
183, which is preferably touch-sensitive includes a number of controls on the instrument 
housing 184 and keys (not shown) as previously described. In another embodiment, the 
instrument 180 can selectively utilize data, siich as to combine audio and ahnotative data, 
without reliance on video data for those ^plications which do not necessarily require this 
form of input. 

In brief, the data files are transmitted &om the local computer station 190 (though 
alternately data could conceivably be transferred directly to the cradle 1 86) to a network 
center 200 which includes a number of remote computer stations 210 to which audio data is 
sent for transcription. The computer stations 210 each utilize human transcriptionists and/or 
voice recognition software to create a transcription record which is uploaded back to the local 
computer station 190 in the generated template format. Reports can then be generated which 
can be stored in the local database 190 in the generated template format, and subsequently 
printed or transferred, such as to a CPR (Computerized Patient Record) 214 The system in 
general therefore can create, maintain and update patient files automatically with the files 
containing several different forms of data. 
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Still referring to Fig. 9, and according to this specific embodiment, the instrument 180 
includes a pinned data exchange SCSI or other connector 187 configured for engagement 
with a corresponding port (not shown) located in a receiving cavity 188 of the cradle 186. As 
should be apparent, the form of data transfer is not critical, for example, the data exchange 
5 connector can also be USB or serial, as previously shown in Fig. 8. 

As shown in Fig. 9, the cradle 186 includes a supporting base portion 194 having the 
receiving cavity 1 88 appropriately sized for retaining the lower portion of the diagnostic 
instrument housing 184, shown partially. Preferably,, the base portion 194 can also include a 
separate storage cavity (not shown) or other means for retaining any of the loose 
1 0 interchangeable instrument heads. 

A control section 196 of the cradle 186 includes a synch button or switch 198, as well 
as a plurality of indicator lamps 202 which indicate specific operational features of the cradle 
186, such as to indicate the charging status of the instrument batteries, the status of data 
transfer, and overall powering of the docking station. For puiposes of the present discussion, 
15 and upon proper attachment, activation of the S3mch button 198 causes all audio^ control, 
annotated and video data to be automatically uploaded to the local PC site 1 90. 

The cradle 186 according to the present embodiment is capable of performing 
additional functions. For example, means are provided for recharging the batteries contained 
in a batteiy compartment (not shown) of the instrument 1 80 while nested. Additionally, the 
2 0 instrument 1 80 can also be powered (for example, when battery powo* is low) while attached 
to the cradle 186, through interconnection to a wall outlet or other source of electrical power. 
The control section 196 can also be configured with additional switches (not shown) which 
interconnect with the controller through the data transfer connector of the instrument 180 to 
allow the instrument to be operated directly from the cradle 186. An advantage realized by 

2 5 this form of control is that the instrument 1 80 can be made capable of receiving data from 

other instruments, such as a clinical vital signs monitor, for storage as part of a patient 
protocol. 

Still refening to Fig. 9, the cradle 1 86 also preferably allows connection to a separate 
video monitor 204 or other peripheral device for viewing of the captured images, such as witii 

3 0 other doctors, patients or interested parties. Altemately, the instrummt 180, also allows 

direct connection to the video monitor 204 without requiring direct use of a cradle 186, if 
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desired Corresponding audio and annotation data can be similarly transferred with the video 
data in a manner known in the field. 

Fig. 1 0 illustrates the general relationship between the network center 200 and 
individual local PC sites 190. The network craiter 200 includes a central server 191 which is 
tied remotely through telephone lines using respective modems 218 to any number of 
identified local sites 190. Each of the local PC sites 190 includie storage capacity in the form 
of the local database 192 which allows data to be uploaded and downloaded relative to a 
central relational database 222, see arrows 224 and 226. 

Fig. 1 1 illustrates alternate forms of data transfer between the network center 200 and 
the local physician's PC sites 190. In one preferred method, the individual sites 190 are 
connected into the network center 200 using LAN server connections using a hub 240, each 
being interconnected using Ethernet 242. In another alternate version, the network center 200 
can be interconnected to one or more of the sites 190 via telephone lines using respective 
modems 218. Finally, the network center 200 can include a web-related database 230 
accessible over the Intemet, shown schematically as 234 using an Internet Source Provider 
(ISP), shown as 236. Similar transfer is possible as shown in the embodiment described 
below between the remote transcription sites 210 and the network center 200 according to 
Fig. 17. 

Figs. 14-16 are flowcharts presenting the sequence of steps used in obtaining data, 
transferring data to a local PC site 1 90, for iqjloading stored data to the network center 200, 
for further uploading audio files to a rmiote transcription site and for obtaining transcription 
textual data which can be incorporated iiito a finalized patimt report. More particularly. Fig. 
14 is a generalized flowchart of the procedure. Fig. 15 is a dia^ammatic representation 
indicating data transfer between the diagnostic instrument 180, the docking station (receiving 
cradle) 186, the local PC site 190, the network center 200 and the remote transcription site 
210, and Fig. 16 is an enhanced flowchart of the one shown in Fig. 14 which relates to the 
flow of data between the examination room where the instrument is first used, the doctor's 
office in which the local PC site 190 is located, the network center 200 and remote 
transcription sites 210 and finally a consulting physician's office 250 in which consultation 
data in the form of audio may also be transcribed and incorporated into a patient record. 
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The preferred record management system is herein described with reference to each of 
Figs. 14-16. The description relates specifically to an office visit by a patient, though it will 
be readily apparent that the above could also apply to a typical round or hospital shift or other 
suitable setting in which a single patient or plurality of patients are to be examined. 
5 Initially, the patient chart is pulled firom the files for the physician. A scripted set of 

instructions are preferably loaded into the instrument for initiahzing and using the instrument 
180 with a plurality of patients. A unique ID code is assigned, both to the user (the 
physician) and to the instrument 180 for allowing data to be uniquely uploaded and 
downloaded to the local database 192 and/or central database 222. After loading the user ID, 

10 the instrument n> is preferably encoded automatically and the physician then initiates the 
examination by utilizing the diagnostic instrument 180 by removing the instrument fi-om the 
cradle 186 and attaching the general purpose instrument head 52, Fig. 2, to the fi^ont interface 
48 of the instrument in the manner described previously. Preferably, the cradle 186 recharges 
the batteries of the instrument 1 80 sufficiently to allow several hours of typical use. 

1 5 Dependmg on the particular protocol of the local database 1 92 (that is, the template of 

the patient data folder created by the database), a patient ID and doctor ID are first captured. 
One way of obtaining either ID is by cs^tuiing and storing video image(s) of the printed 
patient chart. The ID may or may not include bar code information. Alternately, a video 
image of the patient can be taken and the appropriate data can be added via annotation, using 

20 the TFT display 1 83, Accoixiing to another alternate embodiment, doctor and patient 
information can be csptured via a menu in the microprocessor software. In yet another 
alternate manner, the physician could enter all patient data into folders loaded on the 
instrument 1 80, using data entered fipom the local PC 190. Each technique is 
diagrammatically in Fig. 15. As in the preceding, images are captured and stored by first 

2 5 activating the instrument 1 80 filing the image to be captured into memory in the integral 
display 84. Actuation of the shutter control button (not shown) of the instrument 180 allows 
each digital image of interest to be stored mto the buffer memory of the digital video 
processing engine 78, Fig. 8. Vital sign and other pertinent patient data would also be added, 
either as a captured video image or be entering the data directly into the internal memory of 

30 the instrument. Data fix>m other instruments or firom other measurements can be entered into 
the instrument 1 80 by a number of known methods. For example, data could be sent using 
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RF or other wireless technologies. Data could also be entered using the keyboard of flie local 
PC site 190, through buttons, or other known input devices on the instrument itself. 

The physician is then ready to begin examinations, such as done on a daily basis in the 
ofBce, for example. As described above, each patient visit is initialized by capturing a video 
image of the patient ID and storing the image into the internal memoiy of the microprocessor 
78, Fig. 8. Additional patient data can then be captured using the instrument and selectively 
any of the interchangeable instrument heads which may be required. The physician can also 
capture audio data pertaining to each captured video image of interest using the microphone 
142, Fig. 8, the video and audio data being available for playback using speaker 144, Fig. 8 or 
annotative data using the TFT display 1 83 at any point during the examination. The 
instrument 180 according to the present embodiment utihzes an internal calendar with date 
stamping to identify the date and time of each captured image. Alternately, this data could 
be entered separately or other data could be entered, such as from an external source, 
including operating instructions, protocol, height and weight data, as well as other pertinent 
information which can be added using the local computer or the network, for example. Each 
new patient requires identification of the new patient ID, as described above. During the 
exam, after the exam, after examining several patients, or at the end of the day, the physician 
can perform his dictation in the usual manner. At the end of the day, or after a determinate 
number of examinations, the inistrument 180 is loaded into the receiving cavity 188 of the 
cradle 186. The software contained within the instrument 180 further preferably allows 
additional data entry for an earlier patient, if desired, such as to include later obtained data 
from another external source, etc. 

Upon loading the diagnostic instrument 1 80 into the receiving cavity 1 88 of the cradle 
186, the synch button 198 is actuated, automatically transfening the stored audio (WAV), 
video, and annotation data files (if any) to the database 192 of the local PC site 190. The 
software provided in the local PC site 190 loads the raw data into a specific template, an 
example of which is shown in Fig. 18. Preferably, a confirmation mdication is provided on 
the display of the local PC 190 to indicate that all images and audio clips have been removed 
from the instrument 1 80. 

In a particular embodiment, the instrument 180 can also include a counter, preferably 
stored in the EEPROM 172, Fig. 8, which counts the number of images taken by the 
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instrument. As one means for preventing imauthorized use and/or theft, the coimter output 
can be automatically checked and reset by the software contained in the local database 192 
when the instrument 180 is synched thereto. Therefore, if the instrument 180 is not synched 
in the prescribed manner, then fiulher use of the instrument would be disabled. In this 
5 manner, unauthorized persons could not access the instrument and, for example, take pictures. 

During the data transfer to the local database 192, the doctor and patient IDs are first 
located and identified. In the case of use of bar codes, the local PC site 1 90 preferably 
includes recognition software which allows identification of the doctor and patient IDs and 
loads the data into an already existing or newly created patient file. Most preferably, the 

1 0 software includes pattern recognition or bar code recognition programs which can detect the 
existence of a bar code or other pattem firom an existing and captured video image and then 
decode the bar code or pattem if such information is present. In the present embodiment, the 
doctor ID or the patient ID may contain a ID or 2p bar code pattern, the determination of 
which engages the following transcription routine. Details relating to the software for 

15 detecting bar code fix)m a digitally c^tured video image is described in greater detail in 
USSN 08/964,341 filed November 4, 1997. The bar code recognition software can also be 
used to control the instrument 180. For example, the software can be used to indicate the 
type of instrument head being used or which anatomy type a physician is examining or 
imaging. 

2 0 Upon identifying the doctor and patient, the software creates a new data folder in the 

event of a previously unlisted patient, or accesses an already existing patient folder by 
comparison to a list stored in memory. Preferably, a security feature is loaded into the logic 
of the local PC site 190 prompting a user identification window and requiring a password be 
entered prior to allowing access to the raw data for reyiew or prior to transferring the data to 
25 the network center 200. The software automatically stores images and other associated data 
input to a tagged file having the ID number or name attributed to it As such, the files can be 
automatically stored without requiring human intervention or assistance. In addition, the files 
can also be tagged for action, such as additional tests, follow-up visits, inoculations, 
prescriptions, or other procedures. 

3 0 After all of the stored data (video, audio, etc.) has been downloaded onto the local 

database 192 and reviewed, an election is made to send all or part of the data, in this 
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embodiment the digital audio files (WAV) files, such as through phone lines as part of a 
LAN connection , by dial-up networking, e-mail transfer or alternately over the Internet by 
known means to the network center 200. Prior to transmitting this data, the WAV. files are 
first previewed, such as by the physician at the local PC site 190, if desired by selection of the 
5 appropriate entry queued at the patient template. Alternately, other image data can be 
archived to the database 222 of the network center 200 while the audio data is being 
transmitted. 

Preferably, the local PC 190 encrypts the data prior to transferring the data to the 
network center 200, where the data will be decrypted using techniques known in the field. 

10 The details of encryption/decryption do not form an essential part of the present invention and 
therefore require no fiirther discussion. The data is transferred between the local PC site 190 
and the network center 200 using the template originally created at the local PC site. Because 
the data is transferred in this format, it is not necessary to send the corresponding video data 
to the network center 200. However, image files may be transferred for data storage 

15 (warehousing) in the central database 222 or for sending referring letters via e-mail or other 
purposes. 

At the network center 200, the audio digital (WAV) files can be transcribed after 
being loaded into a server or other hardware, the center having a plurality of linked computer 
stations 210 using human transcriptionists in combination with an automatic voice 
2 0 recognition (hereinafter referred to as VR) software system, such as Dragon Systems 
Naturally Speaking to develop a database of a doctor's vocabulary. Preferably, the VR 
software contains an adaptation or learning mode which improves the general efficiency of 
transcription as related to specific physician(s). That is, as the number of transcriptions using 
the dedicated physician necessarily improves more efficiently over time by updating of a 

2 5 specific dedicated physician file. However, unlike traditional uses of voice recognition 

software, the training would be done by transcriptionists rather than the speakers themselves. 

Upon receipt of the raw audio data fi-om the local PC site 190, the physician's ID is 
retrieved Srom memory at the central database 222 and the training file (if existing) is 
accessed. Otherwise, a new doctor's training file is created. An original version of the 

3 0 transcription is then automatically created, the results of which are then subsequently 
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transmitted, also automatically, to a separate PC site 210 for review by a human 
transcriptionist. 

Using the WAV files obtained Scorn memory by accessing the training folder, the 
transcriptionist can effect any changes which may be required based on a review of the 
5 created transcription, the changes being directly inputted into the record and also into the VR 
software into the training file. The above procedure can then be iterated until the training 
using the VR software has progressed to a given level and a suitable transcription is 
produced. The number of iterations (edits) will significantly decrease with an increasing 
number of transcription files, based on the learning mode, and assuming the physician 

1 0 performs an initial vocabulary building exercise typically required of presently known VR 
software. This improvement creates an increase in efficiency and accuracy after an initial 
learning curve for each physician. At the transcription site 210, the transcriptionist can also 
access the video and annotation files, if transmitted and as needed, to further improve the 
reviewing process. The chief benefit of this sort of training method is that the difficult job of 

1 5 training VR software is done by lower paid personnel which is more efficient, thereby freeing 
the physicians to perform the jobs they were trained for. 

Following the transcription procedure at the network center 200, a copy of the 
transcription is removed from the training file and is attached to the specific patient data file 
in the appropriate location prompted by the incoming template. The data file is then 

2 0 transferred to the local PC site 1 90 in the template format having the transcribed information 
added as shown in Fig. 18 in the vicinity of the corresponding video image. As in the 
preceding, data transfer is performed through connections as shown in Fig. 11, At the local 
PC site 190, the transcription can then be reviewed by the physician where the information 
can be reviewed and edited for accuracy and additional editing, if needed. The physician can 

2 5 also access the image and audio portions of the patient chart from the local database 192, if 
needed, while reviewing the transcription. A finalized copy can be printed and approved or 
signed off by the physician prior to adding a hardcopy of the file to the patient record. 

The appropriate files are originally combined using a data file accumulated prior to 
transcription which is presented using a script template. The template can be reviewed and ^ 

30 the audio information can be accessed by cursor, mouse or keyboard control to icons 

presented adjacent to the video images. The icons access the audio files with the aimotation 
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files being presented along with the video files. After the transcription has been completed, a 
hardcopy with the transcription record added appropriately with the images in place of the 
icons can be printed for placing in the patient file. 

The video images, once received into the system are scanned. Subsequent changes, 
5 such as cropping or aiibnishing, etc may be detected to prevent distortion or falsification of 
records. Any annotations, graphical or text, will be stored as a separate file and non- 
destructively overlaid on the image for viewing purposes. Further, the system preferably 
contains appropriate encryption programs for preventing access to the records by 
unauthorized persons. 

1 0 The data can also be transmitted over the telephone lines in any known manner or via 

the Internet to an EMR 214 or other remote site, with the network center 200 also allowing 
receipt of information therefix)m fiom other sources, etc. to aid in netwoiking. For example, 
information firom an instrument at a remote site relative to the local PC site 190 can be 
transmitted into the local database 192 or be unlinked by known means through the cradle 

15 186 to the instrument 180. 

Though the preceding data management system described in detail, a technique of 
remotely transcribing using a central bank of computers, it should be realized that the 
transcription could c^ainly be done locally. That is to say, the immediate benefit is the 
ability of the instrument of the present inveiition to incorporate multiple forms of data which 

20 can be linked, including audio, video, annotation, etc., to allow data management to be better 
coordinated. The features, though pertaining to the medical profession in the preceding 
embodiment, are clearly applicable to other service providers, including attorneys, insurance 
agents, and the like, as well as a myriad of other suitable applications. 
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WE CLAIM: 

1 1, A metfiod for creating and maintaining records containing at least image and 

2 audio data relating to a plurality of subjects using a hand-held multimedia examination 

3 instnmient having means for capturing various forms of data, the method comprising the 

4 steps of: 

5 capturing image data relating to at least one subject of interest using said hand- 

6 held multimedia instrument; 

7 capturing audio data corresponding to at least one subject of interest using said 

8 hand-held multimedia instrument; 

9 capturing control data relating to at least one subject of interest using said 

10 hand-held multimedia instrument; 

1 1 transferring the captured audio, control and image data from said multimedia 

12 instrument to a computer site; 

13 storing said data in a database associated with said computer site; 

14 transferring audio data from said computer site to a transcription site; 

15 transcribing said audio data at said transcription site mto text-readable data; 

1 6 transfening said text-readable data to said computer database for storage; and 

1 7 incorporating said text-readable data with associated image data as part of a 

18 data record. 

1 2. A method according to Claim 1, including a netwoik center having storage 

2 means, said network center being interconnected to a plurality of transcription sites and at 

3 least one said computer site, the method including the steps of: 

4 transferring audio data from said at least one computer site to said network 

5 center; 



6 storing said audio data in said storage means; 

7 transferring said audio data to a transcription site for transcription; 

8 receiving text-readable data from said transcription site; and 

9 sending said text-readable data to said computer site; and 

10 incorporating said text-readable data with associated image and audio data into 



1 1 said data record in said computer database. 
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1 3. A method according to Claim 1 , wherein said hand-held multimedia 

2 instrument includes means for receiving and storing data from other instruments and 

3 annotation means for annotating onto captured image data, wherein said method includes the 

4 step of transferring all stored data, including annotation data and other instrument data, to 

5 said computer site. 



1 4. A method according to Claim 1, wherem said transcription sites include voice 

2 recognition software capable of automatically transcribing received audio data, said method 

3 including the additional steps of: 

4 utilizing said voice recognition software to transcribe audio data received from said at 

5 least one computer site. 

1 5. A method according to Claim 4, wherein said voice recognition software 

2 includes 

3 learning means based on a specific speaker of said audio data, said method 

4 including the step of creating a computerized training file for each speaker; 

5 identifying and accessing said training file for a specific speaker; 

6 sending the audio data of said identified speaker to a transcription site having 

7 said computerized training file; and 

8 modifying the training file during transcription step using said learning means 

9 to include new vocabulary used by said speaker using the learning means of said voice 

10 recognition software, enabling an increase in the efficiency of subsequent transcriptions of 

11 said speaker. 

1 6. A method according to Claim 1 , wherein said at least one computer site 

2 includes identification means for discriminating object recognizable symbols from a captured 

3 image, said method including the steps of: 

4 scanning stored images received from hand-held multimedia instrument for 

5 object symbology; 
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6 decoding said symbology if detected. 

1 7. A method according to Claim 1 , wherein said hand-held multimedia 

2 instrument is used for medical examinations, the method including the steps of: 

3 initializing at least one patient encounter by creating a data file using said 

4 multimedia instrument; 

5 capturing image, control and audio data relating to said at least one encounter 

6 using said multimedia instrument; 

7 transferring all captured data from said hand-held multimedia instrument to 

8 said computer site; 

9 storing captured data in said computer database; 

10 converting said stored data of each patient encounter into a separate report 

1 1 t^plate having allocation for all captured data based on said created data file; 

12 transferring audio data to a network center having means for identifying and 

13 storing said audio dat£^ 

14 siding identified audio data relating to a user to a transcription site for 

15 transcription; 

16 transcribing said identified audio data into text-readable data; 

17 transferring the text-readable data to said network center, 

18 transferring the text-readable data to said computer site; and 

19 incorporating said text-readable data into the corresponding report template. 

1 8. A method according to Claim 7, including the step of adding other patient- 

2 related data to said data file prior to transferring said captured data firom said hand-held 

3 multimedia instrument to said computer site. 

1 9. A method according to Claim 6, in which said multimedia instrument includes 

2 means for receiving and storing data firom other instruments and means for annotating 

3 captured image data relating to at least one patient of interest, said computer site having 

4 means for storing all transferred data firom said multimedia instrument into said report 

5 template. 
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1 10- A data management system conqjrising: 

2 multimedia examination instrument having imaging means for 

3 capturing at least one image of interest, audio recording means for recording audio data 

4 relating to said at least one image, control data recording means for capturing data necessary 

5 for the management of captured image and audio data, and first storage means for storing 

6 captured image and audio data; and 

7 means for transferring audio data between said first storage means and 

8 transcribing means for converting said audio data into text-readable data. 

1 11. A data management system as recited in Claim 10, wherein said transferring 

2 means includes a computer site havmg second storage means capable of storing all captured 

3 data fiom said hand-held multimedia instrument. 

1 12, A data management system as recited in Claim 10, wherein said computer site 

2 includra identification means for identifying data files created by said multimedia instrument 

3 and means for placing said captured data into a report template stored by said second storage 

4 means, and in which said transfening means includes a network center interconnected to said 

5 computer site, said network routing center interconnecting at least one computea- site with a 

6 pluiaUty of transcription sites, said network center including a computer having centraKzed 

7 storage means. 

1 13. A system according to Qaim 10, in which said hand-held multimedia 

2 instrument includes means for receiving and storing data fiom other instruments and means 

3 for annotating captured image data relating to a subject of interest, said computer site having 

4 means for storing all data transferred &om said hand-held multimedia instrument into a 

5 corresponding report template. 

1 14. A system according to Claim 12, wherein said transcription sites include voice 

2 recognition software enable of automatically producing a transcription record. 
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1 15. A system according to Claim 10, wherein said at least one multimedia 

2 instrument includes means for securing against theft and unauthorized use, said computer site 

3 having means for authenticating control data transferred from said at least one hand-held 

4 multimedia instrument and for disabling said at least one instrument upon detection of an 
unauthorized use. 

1 16. A system according to Claim 15, wherein said at least one multimedia 

2 instrument includes means for counting the number of images captured, said disabling means 

3 being connected with said counting means to disable said instrument after a predetermined 

4 number of images have been taken. 

1 17. A system according to Claim 12, wherein each of said transcription sites 

2 include a plurality of training files, each said training file relating to a specific speaker, said 

3 network center having means for identifying the speaker of audio data received from said at 

4 least one computer site and for transferring audio files of said speaker to a transcription site 

5 having a training file corresponding to said speaker. 

1 1 8, A system according to Claim 1 1 , wherein said transferring means includes 

2 means for capturing audio and control data obtained over a telephone line as a second storage 

3 means redundant to said instrument, said transferring means fiirther being capable of sending 

4 audio and control data needed to identify subjects of said instrument obtained by said second 

5 storage means to said computer site. 

1 19. A system according to Claim 1 1 , including template means forming patient 

2 records for retaining stored daU from said at least one multimedia instrument, said 

3 transcription means including means for transferring data to selective fields of selective 

4 patient records created by said template means. 

1 20. A system according to Claim 1 1 , wherein said transferring means includes 

2 means for automatically transferring audio, voice and other stored data to a plurality of 

3 remote locations for consultation upon transfer of said stored data to said first storage means. 
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21. A system according to Claim 1 1 , wherein said transferring means includes 
means for transferring stored data between other multimedia instruments. 



wo 99/42029 



1/16 



PCTAJS99/03559 




SUBS lllUlb SHEET (RULE 26) 



wo 99/42029 



PCT/OS99/03559 



2/16 




SUBSTITUTE SHEET (RULE 26) 



wo 99/42029 



PCTAJS99/03559 




SUBSTTTUTE SHEET (RULE 26) 



wo 99/42029 



PCT/US99/03559 




SUBSTITUTE SHEET (RULE 26) 



wo 99/42029 



PCTAJS99/03559 



r 



O 

O 



PI 

CNI 



I CO 



UJ 

CO 



5/16 



<: 



5 



o 

Q UJ 
<C O 
CL. 



CO 



O 



CO 
CO 



ro «r- CM 

CO CO CO 

ITT 

CM 

^ ^ ^ 

CM m Q:: 

CO CO UJ 
ID 



oo 
o 
iZ 



L_T_J 



OO 



o 



CO 

to 



CO 

1 



CO 



9 



V 



o 
in 



o 

fZ UJ 



o 
o 



o " 
UJ z 

I z 

^ CO 
O UJ 



OO 

ro 



V 











UJ 


1— 


UJ 




CD 



r 



-4- 




O UJ 

gixji^ 

q2 



O 
UJ 
CD 



o 

CO 



CL 



Q 



"n 



CM 

in 



CO 

in 



> o 

o 
o 



1_ 



or: ^ 

UJ 



a. CO 



SUBSTITUTE SHEET (RULE 26) 



wo 99/42029 



PCTAJS99/03559 




SUBSTrrUTE SHEET (RULE 26) 



wo 99/42029 



PCT/US99/03559 




SUBSTITUTE SHEET (RULE 26) 



wo 99/42029 



PCT/US99/03559 




SUBSTITUTE SHEET (RULE 26) 



wo 99/42029 



PCT/US99/03559 



9/16 



COMPUTER RECORDS 

DELETED 



PAPERWORK 
LOST 



PATIENT CHART 
FROM FILES 



PHYSICIAN EXAMINES 
PATIENT 






PHYSICIAN 1 
EXAMINA 
DICT/ 


DOCUMENTS 
riON VIA 

moN 



TAPES LOST 
OR BAD 



PHYSICIAN RLLS 
IN CORRECTIONS. 
SIGNATURE 



CHART NOTES TO 
PHYSICIAN FOR 
REVIEW, 
. SIGNATURE 



PRINT OUT 
CHART NOTES 



AUDIOTAPE TO 
TRANSCRIPTIONIST 



TRANSCRIPTIONIST 
TYPES CHART 
NOTES 



TAPE ERASED 



nG.i2 

Prior Art 



SUBSTITUTE SHEET (RULE 26) 



wo 99/42029 



PCT/US99/03559 



10/16 



O Q- 
CO 




m 

LU CO 
CO z 



to 
on 

is 

S to 
o o 

o o 

CO 



or: 



i 




DIO 




AU 


o 



Qg CO 
O LU 
O 



a. 

ct: 



o 

& 

o 
o 

en 
o 










t 
o 


CO 


\ 




EFILE IN 


:CORDS 




t-LJ 


1 • I 


:nts 


QUEUI 


PATIE 



o 
□_ 




SUBSTITUTE SHEET (RULE 26) 



wo 99/42029 



PCTAJS99/03559 



11/16 



= O (y^ 

^ Z r-j 

S ^ <^ 
^ 



CO 












o 




CJ) 




LU 


a: 






Ol 


LU 


UJ 
















o 




CJ> 





O O Q 

Q- § 

O 

^ ^ s 

O = LU 
UJ O 

^ fc= > 
^ % P 

o 



CO 
i_ UJ 

§i 

is 

ct: ^ 
Q- in 
o 



c/^ ^ ^ 

LU Ci Z rv: 

Zj UJ O S 

»^ ^ f= ^ 

o H-" ^ -< 

U- Ql r_> 
1^ CT) C_> > ^ 



Q CO O 
3 5 ^ LU 

^ 5 »— ct: 



^ O ZD 5 
CO I— ^ 



CO 



u z o 
-So 

LU Z > 

q:: zs UJ < . 
p hz ^ LU 

o CO S 3 



O 

iZ 



CO 



o 

i 

o 



go 



a. 



O UJ 

UJ bn 

Q CO 

Q o 
of ^ 

= ^ 
CO o 



SUB SlllUlk SHEET (RULE 26) 



wo 99/42029 



PCT/US99/03559 





o 




(ARE 


ECTIN 


ION 












SOI 


QZ 




O 


o 


O 


to 




1 . 1 


i 


TRAI 


WHILI 


J— 



o zz. 



12/16 



S CD 



o 
o 




ct: 



o 
o 



LU ^ to 
Q O LU 

CP g Z 











O 


1 — 


S 




to 







<c. 













^— 




z 
til 


PHYSi 




PATH 




O 
QZ 

5 



Z Q 

^ o 
o o 

ii 

a. 



SUBSTITUTE SHEET (RULE 26) 



wo 99/42029 



PCT/US99/03559 



13/16 



FXAMINATION ROOM 



DOCTOR IDS 
SELF VIA 
AUDIO 



DOCTOR USES INSTRUMENT 
TO TAKE BARCODE IMAGE" 
AT START OF DAY 
(OR STORES BARCODE DATA 
OF INDIVIDUAL FROM 
PATIENT OR CHART) 

ZZT 



DOCTOR CAPTURES 
IMAGE OF PATIENT 
BARCODE (ID) 



DOCTOR TAKES 
BARCODE IMAGES OF 

COMMANDS OR 
UTIUZES INSTRUMENT 
PROTOCOL 

■ I 



DOCTOR CAPTURES 
IMAGES OF PATIENT 
ANATOMY USING 
SELECTIVE INSTRUMENT 
HEA DS AND BARCODES 



DOCTOR SELECTIVELY 
DICTATES, AUDIO. 
ANNOTATIVE DATA 



I 



AS PATIENTS SHIFT, 
INSTRUMENT IS PLACED 
IN RECIEVING STATION 
TO RECH ARGE BATTERIES 



DOCTOR EDITS FILES 
(ADDITION OF TEXT, VOICE, 
ANNOTATIVE DATA) 



nG.16 



FIG. 


RG. 


16A 


16B 



DOCTOR OFFICE 



LOCAL PC SITE 
SCANS IMAGES FOR 
BARCODES 






IMAGES DELETED 
FROM INSTRUMENT 


1 





COMPUTER CALLS 
NETWORK CENTER 
TO TRANSFER 
DIGmZED AUDIO 

FILES FOR 
TRANSCRIPTION 



ENCRYPTION 



LOCAL PC SOFTWARE 
COMBINES IMAGE, TEXT 
ANNOTATIVE. VOICE AND 
OTHER DATA INTO WORD 

OR OTHER TEMPUTE 



I 



DOCTOR INFORMED OF 
COMPLETED 
TRANSCRIPTIONS 

I 





DOCTOR OR NURSE 
PRACTmONER REVIEWS 

NOTES AGAINST 
TRANSCRIPTION RESULTS 












SEND 
INFORMATION 
TO EMR 




PRINT OUT 
DOCUMENT 
FILE 



SUBSTITUTE SHEET (RULE 26) 



wo 99/42029 



PCT/US99/03559 



U/16 

NETWORK CENTER SEPERATE PHYSICIAN 
AND TRANSCRIPTION SITES REMOTE SITE 



FILE ROUTING TO 
TRANSCRIPTION SITE 



OBTAIN ID OF 
USER/DOCTOR AND PULL 
DOCTOR'S PREVIOUS 
VOICE RECOGNITION 
SOFTWARE TRAINING RLE 



AUTOMATIC TRANSCRIPTION 
USING VOICE RECOGNITION 
SOFTWARE 



HUMAN TRANSCRIPTIONIST 
TRAINS VOICE RECOGNITION 
SOFTWARE SYSTEM TO 
DOCTOR'S VOICE WHILE' 
COMPLETING AND REVIEWING 
RESULTS OF AUTOMATIC 
TRANSCRIPTION 



UPLOAD TEXT FILES 
TO NETWORK CENTER 
AND LOCAL PC SITE 
WFTH TAGGING OF 
TEXT FILE TO WAV FILE 



IF CONSULTATION 
REQUIRED, SEND TO 
APPROPRIATE PHYSICIAN 



DIGITIZED VOICE 
RECORD OF 
CONSULTATION 
DOCTOR 



STORE UPDATED 
VOICE RECOGNITION 
TRAINING FILE 



SUBSTITUTE SHEET (RULE 26) 



wo 99/42029 



16/16 



PCT/US99/035S9 



WelcJ^AUyn 



1 



8424 



\ 



Wave Sound 

Audio 1: 
2/3/98 
1:15:56 PM 
47 seconds 



■4? 

Wave Sound 

Audio 2: 
2/3/98 
1:15:56 PM 
34 seconds 



) Patient name: Madeleine Perkins 

Patient ID: 314-15-9527 

!|f Providername: DrVitoLosito 

Exam date: 3 February, 1 998 

1:15:56 PM Patient complained of unsightly birth- 
mark on palm of right hand. It appears to be a benign 
hemangioma. Newly formed blood vessels arc readily 
apparent in the bright, protruding and sharply demarcated 
lesion. Since the patient has l\ad for many years, we will 
evaluate in six months. 



1 :43:30 PM BP 126/86, P 82, Wt 1 90, HEENT: 
PERJILA, EOMs intact, TMs nl, oropharynx benign. 
NECK.: Supple w/o JVD, bruits, or thyromegaJy. 
CHEST: BS clr to percussion and auscultation. HEART: 
WNL w/o gallop, murmur, rub, click or irregularity, 
EXT: w/o edema, pulses intact 




(^Approved: 



Image 2: ifUght TM, normal 



Vilo Losito MD 
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